Interface apparatus and method for transmitting and receiving media data

ABSTRACT

An interface is provided for providing a media service. The interface includes an encapsulation layer for encapsulating coded media data, a delivery layer for transmitting the encapsulated media data to another entity, and a control layer for controlling transmission of the media data.

PRIORITY

This application is a Continuation Application of U.S. Pat. No.8,638,818, issued on Jan. 28, 2014, which claims priority to anapplication entitled “Interface Apparatus and Method for Transmittingand Receiving Media Data” filed in the Korean Industrial Property Officeon Apr. 20, 2010, and assigned Serial No. 10-2010-0036180, the contentsof which are hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention generally relates to an apparatus and method fortransmitting and receiving multimedia data, and more particularly to aninterface apparatus and method for encapsulating various types ofmultimedia data and transmitting and receiving the encapsulatedmultimedia data between devices on a network aimed at multimediatransmission.

2. Description of the Related Art

For efficient transmission of various multimedia contents, MPEG (MovingPicture Experts Group) has developed a variety of standards, such asMPEG2 TS (Transport Stream) and MP4 systems and file formats. Further,standards organizations, such as IETF (Internet Engineering Task Force)and IEEE (Institute of Electrical and Electronic Engineers) 3GPP (3^(rd)Generation Partnership Project), have developed a variety of protocolsfor transmission of various multimedia content on the Internet, andplatform technologies for configuring a multimedia content use the MPEGstandards.

As the trend toward convergence of broadcast and communication serviceshas continued since 2000, an environment has emerged where variouscontent and services must be transmitted through different networks.Further, due to advances in content compression and transmissiontechnologies, and improvements in display performance (e.g. TV), usersdesire to use high-definition data at a lower cost.

In order to meet the above requirements, MPEG intends to develop astandard technology that enables efficient transmission of ahigh-definition multimedia content, converged transmission of variousmultimedia contents, and use of various networks in transmission of amultimedia content.

FIG. 1 illustrates a conventional multimedia content transmissionstructure in which different protocols are used according to variousmultimedia content.

Referring to FIG. 1, each application (e.g. e-mail, file transfer, webservice, etc.) is conventionally serviced in a corresponding layer byusing a separate protocol. For example, an e-mail service 101 requiresthe SMTP protocol 103 in the application layer, requires the TCPprotocol 105 in the transport layer, and requires the IP protocol 107 inthe network layer. To the contrary, a voice service 113 requires the RTPprotocol 115 in the application layer, and requires the UDP protocol 117in the transport layer.

For example, assume that movie content is provided to a user. One frameconstituting the movie content typically includes a video stream, anaudio stream, and caption information. In the case of using the protocolstructure of FIG. 1, the video stream, the audio stream, and the captioninformation must be transmitted using different protocols, as mentionedabove. Therefore, when the conventional protocol structure is used,content including various types of media (hereinafter referred to as“multimedia content”), such as movie content, is inefficient totransmit. That is, in the method in current use for providing multimediacontent, separate protocols must be applied according to respectivemedia constituting each multimedia content, which results ininefficiency in multimedia content transmission. In particular, there isdifficulty in transmission of complex multimedia content includingvarious multimedia contents.

SUMMARY OF THE INVENTION

Accordingly, the present invention has been made to solve theabove-mentioned problems occurring in the prior art, and an aspect ofthe present invention provides an apparatus and method for transmittingmultimedia data including various types of multimedia content by usingvarious types of networks.

Further, another aspect of the present invention provides an interfacefor integrated transmission of multimedia content.

Further, yet another aspect of the present invention provides a packetformat for integrated transmission of multimedia content.

In accordance with an aspect of the present invention, an interface isprovided for providing a media service. The interface includes anencapsulation layer for encapsulating coded media data, a delivery layerfor transmitting the encapsulated media data to another entity, and acontrol layer for controlling transmission of the media data.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of the presentinvention will be more apparent from the following detailed descriptiontaken in conjunction with the accompanying drawings, in which:

FIG. 1 illustrates a conventional protocol structure in which differentprotocols are used according to various application services;

FIG. 2 illustrates three layers subdivided for explaining necessaryfunctions and interfaces in accordance with an embodiment of the presentinvention;

FIG. 3 illustrates a structure of an IP packet of multimedia transmittedusing MMT with a structure as shown in FIG. 2;

FIG. 4 illustrates an example of actually providing a user with aservice by using an MMT system as defined in FIG. 2;

FIG. 5 illustrates a service in the form of a transmission file format,which is transmitted using a multimedia system with a structure as shownin FIG. 2; and

FIG. 6 illustrates a packet structure of a content transmitted using amultimedia system with a structure as shown in FIG. 2.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE PRESENT INVENTION

Hereinafter, embodiments of the present invention will be described withreference to the accompanying drawings. In the following description,the same elements will be designated by the same reference numeralsalthough they are shown in different drawings. Further, various specificdefinitions found in the following description, such as specific valuesof packet identifications, contents of displayed information, etc., areprovided only to help the general understanding of the presentinvention, and it will be apparent to those skilled in the art that thepresent invention can be implemented without such definitions. Further,in the following description of the present invention, a detaileddescription of known functions and configurations incorporated hereinwill be omitted when it may make the subject matter of the presentinvention rather unclear.

The present invention defines an interface for integrated transmissionand reception of multimedia content. The interface defined in thepresent invention provides three functional layers. A first functionallayer is an encapsulation layer, a second functional layer is a deliverylayer, and a third functional layer is a control layer.

The encapsulation layer processes and encapsulates various types ofcoded multimedia content received from a media coding layer. Further,the encapsulation layer appropriately converts the encapsulatedmultimedia content to a format storable in a storage device or a formattransmittable to another entity or a user equipment on a network throughthe delivery layer.

The delivery layer defines functions for providing a transmissionenvironment optimized for the multimedia content in order to transmitthe encapsulated multimedia content through various networks.

The control layer defines consumption information (e.g. electronicprogram guide (EPG)) required for a user to consume the multimediacontent and functions required for the delivery layer to efficientlytransmit the multimedia content.

FIG. 2 illustrates the above three functional layers that are subdividedon a protocol-by-protocol basis according to an embodiment of thepresent invention.

In the present invention, the encapsulation layer includes three layersof E.1 211, E.2 212, and E.3 213, the delivery layer includes threelayers of D.1 221, D.2 222, and D.3 223, and the control layer includesthree layers of C.1 231, C.2 232, and C.3 233. In FIG. 2, layerscollectively designated by reference numeral “270”, that is, fragmentNAL unit, AU, MPEG-2 TS, RTP/HTTP, and UDP/TCP are depicted only for thebetter understanding of the layers according to the present invention,and therefore it should be noted that the functions of the MMT E.3 layer213, MMT E.2 layer 212, MMT E.1 layer 211, MMT D.1 layer 221, and MMTD.2 layer 222 do not necessarily coincide with those of the fragment NALunit, AU, MPEG-2 TS, RTP/HTTP, and UDP/TCP layers.

Referring to FIG. 2, the media coding layer 200 generates media datafragments (“fragments”) by dividing multimedia content in units offragments and coding the divided fragments. Further, the media codinglayer 200 generates fragment-related information. This fragment-relatedinformation is information representing a relation between the generatedfragments. For reference, a combination of a given number of fragmentsconstitutes an access unit, and a combination of a given number ofaccess units and access unit-related information constitutes anelementary stream.

The encapsulation layer has the following configurations and functions.

The E.3 layer 213 of the encapsulation layer receives fragments,fragment headers, and fragment-related information from the media codinglayer 200, and generates access units by using the received fragments,fragment headers, and fragment-related information. Further, the E.3layer 213 generates access unit-related information that is informationon a relation between the generated access units.

The access unit-related information refers to, for example, a decodingtime, a composition time, a size, an access sequence number, and thelike. The generated access units and access unit-related information aretransferred to the E.2 layer 212. For reference, the fragment may be alltypes of data by which to compose a multimedia content, such as dataoutput from a voice codec, data output from a video codec, scenecomposition information for supporting rich media capable of additionalfunctions compared to existing media video, various applications, andfile data.

The E.2 layer 212 generates elementary streams by combining a pluralityof access units and access unit-related information. Further, the E.2layer 212 generates elementary stream-related information. Theelementary stream-related information refers to, for example, the typeof a stream, a stream identifier, the transmission characteristic valueof a stream, and the like.

The E.1 layer 211 generates composition information for a multimediacontent including the elementary streams and a representativetransmission characteristic value allowing the multimedia content to beappropriately consumed. The composition information may be, for example,information necessary when video data and audio data are combined inorder to reproduce a multimedia content including the video data and theaudio data, and the representative transmission characteristic value maybe, for example, a QoS (Quality of Service) that can be expressed by atransmission error rate, speed, and the like.

The delivery layer has the following configurations and functions.

The D.1 layer 221, which corresponds to a transmission protocol similarto the existing RTP or HTTP transmission protocol for multimedia datatransmission in the conventional protocol structure of FIG. 1, is anapplication protocol newly provided by the present invention. In orderto efficiently transmit multimedia content, the D.1 layer 221 generatesnetwork information necessary for multimedia transmission andinformation on characteristics of the multimedia content.

The D.2 layer 222, which corresponds to a network transmission protocolsimilar to the existing UDP or TCP protocol of FIG. 1, generatesinformation for performing flow control and error correction functionsin consideration of various network types.

The D.3 layer 223 provides functions involved in informationtransmission between all the layers, as shown in FIG. 2. That is, theD.3 layer 223 enables multimedia content transfer and eventual necessarycommunication between the layers in order to optimize the generation,transfer, and consumption of a multimedia content. The D.3 layer 223performs operations for satisfying the Quality of Experience (QoE) ofmultimedia content, generates information for the these operations, andif necessary, transmits the information to an entity that receives themultimedia content. Further, the D.3 layer 223 allows the multimediacontent to satisfy its QoS in the delivery layer, and to this end,performs communication with all the layers of FIG. 2.

The control layer has the following configurations and functions. Thecontrol layer is divided into three layers according to target layersfor control.

The C.1 layer 231 generates information necessary for the transfer andconsumption of multimedia content by using information generated in theencapsulation layer E.1 211, E.2 212, E.3 213, and provides thegenerated information to a user equipment. The information necessary forthe transfer and consumption of multimedia content refers to, forexample, information necessary for content consumption, such as thetypes of content constituting the multimedia, a relation between thecontent, and the consumption order of the content, and discoveryinformation through which the multimedia content can be discovered.

The C.2 layer 232 provides the D.1 layer 221 and the D.2 layer 222 withcontrol functions necessary for media transmission. For example, the C.2layer 232 controls the D.1 layer 221 to perform frame configurationconsidering the multimedia content. With regard to this, the C.2 layer232 performs control for the D.1 layer 221 in consideration of the casewhere the multimedia content is transmitted through various networks.Further, the C.2 layer 232 provides the D.2 layer 222 with the flowcontrol of a session for transmitting a multimedia content.

The C.3 layer 233 accesses and controls layers lower than the deliverylayer, that is, an IP (Internet Protocol) layer 240, a data link layer250, or a physical layer 260. For example, the C.3 layer 233 receivesnetwork transmission delay measurement information from the data linklayer 250, and allows the delivery layer to appropriately transmitmultimedia content by using the received information. Further, the C.3layer 233 may receive transmission error rate information from thephysical layer 260, and perform appropriate control necessary to reducethe transmission error rate. An example of the control for reducing thetransmission error rate may be instructions to retransmit a transmittedpacket where an error occurs.

FIG. 3 illustrates a structure of an IP packet of multimedia transmittedusing an MPEG Media Transport (MMT) interface with a structure as shownin FIG. 2.

The information required to operate each layer in FIG. 2 is included ineach corresponding layer head constituting the payload of the IP packet300 of FIG. 3. In FIG. 3, the IP packet 300 includes an IP header 301and a payload 310. The payload 310 includes each layer header in whichthe information generated in each of the above-mentioned layers isincluded.

More specially, media coded data generated in the media coding layer isincluded in the E.3 layer payloads 307, 309 in units of access units.The E3 layer headers 306, 308 include access unit-related informationgenerated in the corresponding layers, the E.2 layer header 305 includeselementary stream-relayed information generated in the correspondinglayer, and the E.1 layer header 304 includes composition information anda representative transmission characteristic value generated in thecorresponding layer. In addition, the D.1 layer header 303 includesinformation on the characteristics of the multimedia content, which isgenerated in the corresponding layer, and the D.2 layer header 302includes information for flow control and error correction.

In the present invention, since necessary information is transferredusing the IP packet structure shown in FIG. 3, a network entitysupporting a specific layer can acquire information necessary forfunctions which are to be performed in the corresponding layer, from thecorresponding layer header without accessing its payload.

FIG. 4 illustrates an example of actually providing a user with aservice by using an MMT system according to an embodiment of the presentinvention.

This example of FIG. 4 corresponds to an example of providing a servicefor transmitting advanced video codec (AVC) data on an HTTP protocol,and shows the case where the service can be provided using only the E.1layer 411 and the C.1 layer 431. In this way, the present inventionmakes it possible to provide a service by using only some of the layersdefined in FIG. 2 to provide a user with the corresponding service.

FIG. 5 illustrates a service in the form of a transmission file format,which is transmitted using a multimedia system according to anembodiment of the present invention.

In the E.3 layer 513, coded media data 514 is defragmented in units ofaccess units. This is shown by the E.3 boxes in FIG. 5.

The E.3 layer data is stored in the E.2 layer 512, and the E.1 layer 511includes composition information for multimedia content, which isinformation necessary for the D and C layers to read information relatedto a service or session including elementary streams and provide it as aservice, and a representative transmission characteristic value allowingthe multimedia content to be appropriately consumed. Since a user mayaccess a multimedia service from any point of time after thecorresponding multimedia service is initiated, as is the case where auser turns on a TV terminal in the middle of providing the correspondingmultimedia service, the composition information and the representativetransmission characteristic value need to be repeatedly expressed.

Among layers lower than the E.1 layer 511, the D.1 layer 521 includesnetwork information necessary for transmission of a corresponding packetand information on the characteristics of a corresponding multimediacontent, and the D.2 layer 522 includes network information necessaryfor transmission using the IP layer and information for flow control anderror correction.

FIG. 6 illustrates a packet structure of media content data transmittedusing a multimedia system according to an embodiment of the presentinvention.

In FIG. 6, a media slice 601 sliced to an appropriate size according tothe type and attribute of content data is packetized as the payload ofan E.3 packet 602. Further, the E.3 packet 602 is packetized by an E.2packet 603 while being divided in units of access units. Further, theE.2 packet 603 is packetized by an E.1 packet 604 while being divided inunits of elementary streams. The E.1 packet 604, together withinformation for transmission using the TCP or UDP layer and informationon the characteristics of a multimedia content, is packetized by a D.1packet 605. The D.1 packet 605, together with information necessary fortransmission using the IP layer and information for flow control anderror correction, is packetized by a D.2 packet 606. Finally, the D.2packet 606 is packetized by a conventional IP packet 607.

According to the present invention as described above, variousmultimedia content can be optimally transmitted at a minimized cost byusing a converged framework. Accordingly, since not only can a user beefficiently provided with multimedia-related content, but providers ofthe multimedia content can also provide services through one convergedsystem, efficiency and flexibility in providing the services can beimproved.

While the invention has been shown and described with reference tocertain embodiments thereof, it will be understood by those skilled inthe art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the invention as definedby the appended claims.

What is claimed is:
 1. A method for transmitting multimedia content, themethod comprising: generating an access unit including media fragmentsand first information related to the multimedia content; generatingsecond information for delivering the multimedia content, the secondinformation comprising information related to a consumption order ofcontent constituting the multimedia content and information throughwhich the multimedia content can be discovered, wherein the multimediacontent comprises a stream including the access unit; and transmittingthe access unit and the second information, wherein the firstinformation comprises composition information for combining video dataand audio data composing the multimedia content and a representativetransmission characteristic value including quality of service (QoS). 2.The method of claim 1, wherein transmitting the access unit and thesecond information comprises: transmitting the access unit correspondingto a protocol, wherein a packet which is based on the protocol includesinformation related to error correction.